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- The MAILING DATE of this communication appears on the cover sheet with the correspondence address 
Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 1 33). 

- Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1 .704(b). 

Status 

1 )K Responsive to communication(s) filed on 22 March 2004 . 
2a)S This action is FINAL. 2b)D This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 1 1 , 453 O.G. 213. 

Disposition of Claims 

4) ^ Claim(s) 7-30 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) ISI Claim(s) 3-5.27-31.35-37 and 40-46 is/are allowed. 

6) £3 Claim(s) 1.2.6-26.32-34.38 and 39 is/are rejected. 

7) D Claim(s) is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) D The specification is objected to by the Examiner. 

10)E3 The drawing(s) filed on 22 March 2004 is/are: a)K accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85(a). 

Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 
1 1 )D The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-1 52. 
Priority under 35 U.S.C. §§119 and 120 

12)D Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 19(a)-(d) or (f). 
a)DAII b)D Some*c)D None of: 

Certified copies of the priority documents have been received. 

Certified copies of the priority documents have been received in Application No. . 



1. D 

2. D 

3. Q 



Copies of the certified copies of the priority documents have been received in this National Stage 
application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 

13) Q Acknowledgment is made of a claim for domestic priority under 35 U.S.C. § 1 19(e) (to a provisional application) 

since a specific reference was included in the first sentence of the specification or in an Application Data Sheet. 
37 CFR 1.78. 

a) □ The translation of the foreign language provisional application has been received. 

14) D Acknowledgment is made of a claim for domestic priority under 35 U.S.C. §§ 120 and/or 121 since a specific 

reference was included in the first sentence of the specification or in an Application Data Sheet. 37 CFR 1 .78. 



Attach ment(s) 

1) £3 Notice of References Cited (PTO-892) 4) □ Interview Summary (PTO-413) Paper No(s). 

2) □ Notice of Draftsperson's Patent Drawing Review (PTO-948) 5) □ Notice of Informal Patent Application (PTO-1 52) 

3) □ Information Disclosure Statement(s) (PTO-1449) Paper No(s) . 6) □ Other: 
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DETAILED ACTION 



1 . The text of those sections of Title 35, U.S. Code not included in this action can 
be found in a prior Office action. 



2. The filing of 22 March 2004 was entered to the following effect: 

- The title was changed as indicated and the objection is withdrawn as satisfied. 

- The specification was changed as indicated and the objections are withdrawn. 

- The claims were changed as indicated and the objections have been withdrawn. 

- Claims 31-46 were added and all claims were examined on the merits. 



3. The proposed replacement drawings were received on 22 March 2004. 
These drawings are accepted and the objections are withdrawn as satisfied. 



Response to Amendment 



Drawings 



Claim Rejections - 35 USC § 103 



4. This application currently names joint inventors. In considering patentability of 
the claims under 35 U.S.C. 103(a), the examiner presumes that the subject matter of 
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the various claims was commonly owned at the time any inventions covered therein 
were made absent any evidence to the contrary. Applicant is advised of the obligation 
under 37 CFR 1 .56 to point out the inventor and invention dates of each claim that was 
not commonly owned at the time a later invention was made in order for the examiner to 
consider the applicability of 35 U.S.C. 103(c) and potential 35 U.S.C. 1 02(e), (f) or (g) 
prior art under 35 U.S.C. 103(a). 

Kenne et a/, Hsu et al & Guerreri 

5. Claims 1, 2, 6, 7, 9, 10, 13-16, 18, 19, 21, 23, 24 & 32 are rejected under 35 
U.S.C. 103(a) as being unpatentable over Kenne et al ("Hybrid Language Models and 
Spontaneous Legal Discourse", 4 th International Conference on Spoken Language, 
October 1996) in view of Hsu et al (U.S. Patent 5,677,991 A) and further in view of 
Guerreri (U.S. Patent 5,189,727 A). 

6. Regarding claims 1 , 13, 18 & 23; Kenne et al in describing hybrid language 
models reads on the features of the claim for selecting a recognizer from a number of 
recognizers as follows: 

- Kenne et al (with the "transcripts" input on page 717, section 2, Data - in the right 
column) reads on the feature of a) receiving an input stream; 

- Kenne et al (with the "perplexity" of the 2 nd paragraph from end of left column to the 
top of the right column) reads on the feature of b) deriving selection information, 
where the selection information includes performance-related information; 
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- Kerine et al (with the "switching" of section 4, Results - in the right column page 
718) reads on the feature of c) using the selection information to select results from 
at least one enabled recognizer; 

- Kenne et al does not mention applications. Hsu et al (with the invention for flexible 
speech recognition that selects isolated word speech or continuous speech 
recognizers) reads on the feature of d) returning the results to the application (61 in 
figure 2). It would have been obvious to a person of ordinary skill in the art of 
speech signal processing at the time of the invention to apply the method/teachings 
of Hsu et al to the device/method of Kenne et ai providing a structure to put the 
strategy for selecting between recognizers into practice. 

- With specific regard to claim 18, the intrusion avoidance of Hsu et al (51 in figure 2) 
corresponds to and performs the function of the predictor of the claim. 

With further regard to the feature added to claims 1 C) , 13 c >, 18 c > and 23 c >, 18 C ), 
neither Hsu et al nor Kenne ef al mention that the recognizer is enabled based on 
expected future performance. Guerreri , with the invention of a method and apparatus 
for language and speaker recognition terms setting pattern recognition preferences 
teaches the "future selection" technique that specifically enables the recognizer (column 
4 lines 46-54). It would have been obvious to a person of ordinary skill in the art of 
speech signal processing at the time of the invention to apply the method/teachings of 
Guerreri to the device/method of Hsu et al & Kenne et al because the objective of 
selecting one recognizer (over others) is to improve performance. 
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7. Regarding claim 2, the claim is set forth with the same limits as claim 1 . 
Kenne et al (lines 8-1 1 of section 4, Results in right column page 718) teaches the 
feature that causes a recognizer to be selected that is different than the recognizer used 
in a previous interaction (by virtue of the fact that a switch is made only if the perplexity 
changes, which would occur only if the current interaction differs from previous 
perplexities). 

Neither Kenne et al nor Guerreri disclose that selection information is updated; 
Hsu et al (lines 57-60) teaches this by setting the reference score as a baseline that is 
necessary to provide further evaluation. It would have been obvious to a person of 
ordinary skill in the art of speech signal processing at the time of the invention to apply 
the method and/or teachings of Hsu et al to the device/method of Kenne et al & Guerreri 
as a conventional programming method of preserving prior readings by storing them 
elsewhere to avoid their being overlaid by new measurements using the same algorithm 
code register fields. 

8. Regarding claims 6 and 24; the claims are set forth with the same limits as 
claims 1 and 23, respectively. Kenne et al (in the 2 nd paragraph of Section 4, Results - 
right column page 718) reads on the feature that the enabling information - and that 
consequently the performance-related information - comprises at least one type of 
information from the group comprises of: channel characteristics, device characteristics, 
user information, contextual information, dialog state, recognizer costs and performance 
history. 
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9. Regarding claims 7 and 14; the claims are set forth with the same limits as 
claims 1 and 13, respectively. Kenne et al (in the 1 st line of page 718) records the 
classes of people into separate tracks to distinguish lawyers from witnesses, thereby 
reading on the feature that deriving the selection information further comprises 
analyzing the input stream for channel characteristics. 

10. Regarding claims 9, 15 & 19; the claims are set forth with the same limits as 
claims 1, 13 & 18 respectively. Kenne et al (with Table 2 and the last paragraph of the 
right column, both on page 717) distinguishes between Lawyers and Witnesses 
(Statements & Questions), thereby reading on the feature of receiving contextual 
information associated with the input stream. 

1 1 . Regarding claims 1 0 and 1 6, the claims are set forth with the same limits as 
claims 1 and 13, respectively. Kenne et al (with the "switching" of Section 4, Results - 
in the right column page 718) reads on the feature of receiving recognizer information 
from the enabled recognizers to be used in the selection information. 

12. Regarding claim 21, the claim is set forth with the same limits as claim 18. 
Kenne et al (with the "switching" of section 4, Results - in the right column page 718) 
reads on the feature that the predictor is operable to select a recognizer based upon the 
converted stream. 
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13. Regarding claim 32, the claim is set forth with the same limits as claim 18. 
Kenne et al & Hsu et al do not disclose sources of contextual information. 

Guerreri teaches that feature where contextual information comprises information from 
at least one item of information derived from the set of information comprising 
information related to the environment around the input stream, characteristics of a user 
generating the input stream, information derived from a call using network services, 
gender, age, ethnicity, information relating to the user's first (native) language, personal 
information about the user, channel characteristics and device characteristics (column 1 
lines 1 9-30). It would have been obvious to a person of ordinary skill in the art of 
speech signal processing at the time of the invention to apply the method/teachings of 
Guerreri to the device/method of Kenne et al & Hsu et al so as to identify speakers' 
statements when transcribing conversations. 

Kenne et al, Hsu et al, Guerreri & Waibel et al' 957 

14. Claim 8 is rejected under 35 U.S.C. 103(a) as being unpatentable over Kenne et 
al in view of Hsu et al and further in view of Guerreri and further in view of Waibel et 
al' 957 (U.S. Patent 5,712,957 A). 

1 5. Regarding claim 8, the claim is set forth with the same limits as claim 1 . 
Kenne et al Hsu et al & Guerreri do not mention separate input devices so would not 
distinguish on that basis. The Waibel et al 957 invention of a speech repair and 
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correction method for speech recognition system provides for different inputs (23 & 24 
in figure 1) and consequently, using this information, locates error in hypothesis with 
highest score from list, generates control signals and second list, combines two lists and 
replaces error with hypothesis with highest combined score. 

This reads on the feature of analyzing the input stream for device characteristics, 
which would have made it obvious to a person of ordinary skill in the art of speech 
signal processing at the time of the invention to apply the method/teachings of Waibel et 
al 957 to the device/method of Kenne et al Hsu et al & Guerreri to allow the decision to 
be influenced by recognition that some devices increase the confidence of recognition 
results done on the streams they produce. 

Kenne et al, Hsu etah Guerreri & Kundu 

16. Claims 11-12, 17 & 25-26 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Kenne et al in view of Hsu et al and further in view of Guerreri and 
further in view of Kundu (U.S. Patent 5,924,066 A). 

1 7. Regarding claims 11, 1 7 & 25; the claims are set forth with the same limits as 
claims 1,13 and 23, respectively. Kenne et al Hsu et al & Guerreri do not mention 
feedback. With the invention for classifying a speech signal, Kundu (column 9 lines 49- 
54) reads on the feature of receiving feedback and including the feedback in the 
selection information. 
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ft would have been obvious to a person of ordinary skill in the art of speech 
signal processing at the time of the invention to apply the method/teachings of Kundu to 
the device/method of Kenne et ai Hsu et al & Guerreri to simplify training by including 
learned lessons on the basis of experience rather than attempting to predict all 
anticipated requirements. 

18. Regarding claims 12 and 26; the claims are set forth with the same limits as 
claims 1 1 and 23, respectively. Kenne et al Hsu et al & Guerreri do not mention 
feedback. Kundu (column 9 lines 50-51 ) reads on the feature that feedback is received 
from one of the group comprised of: off-line analysis, user feedback, and feedback from 
the recognizer. 

It would have been obvious to a person of ordinary skill in the art of speech 
signal processing at the time of the invention to apply the method/teachings of Kundu to 
the device/method of Kenne et ai Hsu et al & Guerreri to adjust for variations in 
confidence levels corresponding to changes in the input stream over time and with use. 

Kenne et ai Hsu et ai Guerreri & Waibel et al' 000 

19. Claims 20, 22, 38 & 39 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Kenne et al in view of Hsu et al and further in view of Guerreri and 
further in view of Waibel et a 1 000 (U.S. Patent 5,855,000 A). 
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20. Regarding claim 20, the claim is set forth with the same limits as claim 18. 
Neither Kenne et al Hsu et al nor Guerreri mention confidence levels. Waibel et al' 000 
(column 1 lines 55-67) teach the feature that the recognizers are also operable to 
provide individual-result confidence levels to the predictor. Because the determination 
is to be made between recognizers, it would have made it obvious to a person of 
ordinary skill in the art of speech signal processing at the time of the invention to apply 
the method/teachings of Waibel et al' 000 to the device/method of Kenne et al, Hsu et al & 
Guerreri in order to select the product of the recognizer providing the highest level of 
confidence. 

21. Regarding claim 22, the claim is set forth with the same limits as claim 18. 
Kenne et al . Hsu et al & Guerreri all select after recognition. Waibel et al' 000 (108 in 
figure 3) teaches the feature to select a recognizer prior to the recognizer receiving the 
input stream (column 7 lines 57-60) which would have made it obvious to a person of 
ordinary skill in the art of speech signal processing at the time of the invention to apply 
the method/teachings of Waibel et al' 000 to the device/method of Kenne et a/, Hsu et al & 
Guerreri because there is no need to operate recognition when controlling information 
would override selection. 

22. Regarding claims 38 & 39; the claims are set forth with the same limits as claims 
20 & 23, respectively. Neither Kenne et al Hsu et al nor Guerreri mention confidence 
levels. Waibel et al 000 teaches the feature where the predictor determines, for each 
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recognizer in the system and for each situation, a recognizer-based confidence value 
(column 1 lines 63-64). It would have been obvious to a person of ordinary skill in the 
art of speech signal processing at the time of the invention to apply the 
method/teachings of Waibel et ai 000 \o the device and/or method of Kenne etal f Hsu et 
a[ or Guerreri for making a preliminary selection by simply examining a confidence 
score to avoid the expense of comparing processing results. 

Kenne et a/, Hsu et ah Guerreri & Goldberg et al 

23. Claims 33 and 34 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Kenne et al in view of Hsu et al and further in view of Guerreri and further in view 
of Goldberg et al (U.S. Patent 5,970,446 A). 

24. Regarding claim 33, the claim is set forth with the same limits as claim 32. 
Kenne et a/ , Hsu et al & Guerreri do not disclose dynamically processing contextual 
information. Goldberg et al , with the invention for selective noise/channel/coding 
models and recognizers for automatic speech recognition, teaches the feature where 
the contextual information is obtained dynamically (column 5 lines 3-6). It would have 
been obvious to a person of ordinary skill in the art of speech signal processing at the 
time of the invention to apply the method/teachings of Goldberg et al to the 
device/method of Kenne et a/ , Hsu et al & Guerreri to continue to operate despite 
changes. 
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25. Regarding claim 34, the claim is set forth with the same limits as claim 32. 
Kenne et ai Hsu et al & Guerreri do not disclose dynamically processing contextual 
information. Goldberg et al teaches the feature where the contextual information is 
predetermined (column 4 line 58-64). It would have been obvious to a person of 
ordinary skill in the art of speech signal processing at the time of the invention to apply 
the method and/or teachings of Goldberg et al to the device/method of Kenne et ai Hsu 
et al & Guerreri to repeat the results of previous processing when conditions recur. 



26. Claims 3-5, 27-31 , 35-37, 40-46 are allowed. 

27. As allowable subject matter has been indicated, applicant's reply must either 
comply with all formal requirements or specifically traverse each requirement not 
complied with. See 37 CFR 1.111(b) and MPEP § 707.07(a). 

28. The following is a statement of reasons for the indication of allowable subject 
matter: 

- The present invention is directed to processing speech using one of several 
available recognition devices. 

- Claims 3, 4 and 27 identify the uniquely distinct feature that "the enabling information 
is used to enable a recognizer based upon its expected future performance ". 



Allowable Subject Matter 
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- The* closest prior art of Guerreri and Kenne et al teach that "the enabling information 
is used to enable a recognizer" but, where it would be obvious that the motive to 
enable/select a recognizer would be produce an expected performance/result, 
indications on performance are based on current or past performance and is not 
separately considered or derived as enabling information, as opposed to merely 
selecting information" as distinguished in the application. Consequently, the prior art 
of reference fails to anticipate or render the above underlined limitations obvious. 

- Claims 5, 28-31 , 35-37, 40-46 depend on claims that were allowed. 

Conclusion 

29. Applicant's amendment necessitated the new ground(s) of rejection presented in 
this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP 
§ 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 
CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of 
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the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the date of this final action. 



30. Any inquiry concerning this communication or earlier communications from the 
Examiner should be directed to Daniel A. Nolan at telephone (703) 305-1368 whose 
normal business hours are Mon, Tue, Thu & Fri, from 7 AM to 5 PM. 

If attempts to contact the examiner by telephone are unsuccessful, supervisor 
Richemond Dorvil can be reached at (703)305-9645. 

The fax phone number for Technology Center 2600 is (703)872-9314. Label 
informal and draft communications as "DRAFT" or "PROPOSED", & designate formal 
communications as "EXPEDITED PROCEDURE". Formal response to this action may 
be faxed according to the above instructions, 

or mailed to: Mail Stop AF (or CPA, etc. - see Official Gazette, 04 November 2003) 

P.O. Box 1450 
Alexandria, VA 22313-1450 

or hand-deliver to: Crystal Park 2, 

2121 Crystal Drive, Arlington, VA, 
Sixth Floor (Receptionist). 

Any inquiry of a general nature or relating to the status of this application or 
proceeding should be directed to Technology Center 2600 Customer Service Office at 
telephone number (703) 306-0377. 

Daniel A. Nolan 
Examiner 
Art Unit 2654 

DAN/d 

May 25, 2004 
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